Understanding Ability and Reliability Differences Measured with Count Items: The Distributional Regression Test Model and the Count Latent Regression Model.

In psychology and education, tests (e.g., reading tests) and self-reports (e.g., clinical questionnaires) generate counts, but corresponding Item Response Theory (IRT) methods are underdeveloped compared to binary data. Recent advances include the Two-Parameter Conway-Maxwell-Poisson model (2PCMPM), generalizing Rasch's Poisson Counts Model, with item-specific difficulty, discrimination, and dispersion parameters. Explaining differences in model parameters informs item construction and selection but has received little attention. We introduce two 2PCMPM-based explanatory count IRT models: The Distributional Regression Test Model for item covariates, and the Count Latent Regression Model for (categorical) person covariates. Estimation methods are provided and satisfactory statistical properties are observed in simulations. Two examples illustrate how the models help understand tests and underlying constructs.

The Manic Idea Creator? A Review and Meta-Analysis of the Relationship between Bipolar Disorder and Creative Cognitive Potential.

Even though a relationship between psychopathology and creativity has been postulated since the time of ancient Greece, systematic meta-analyses on this topic are still scarce. Thus, the meta-analysis described here can be considered the first to date that specifically focuses on the relationship between creative potential, as measured by divergent thinking, and bipolar disorder, as opposed to psychopathology in general. An extensive literature search of 4670 screened hits identified 13 suitable studies, including a total of 42 effect sizes and 1857 participants. The random-effects model showed an overall significant, positive, yet diminutively small effect (d = 0.11, 95% CI: [0.002, 0.209], p = 0.045) between divergent thinking and bipolar disorder. A handful of moderators were examined, which revealed a significant moderating effect for bipolar status, as either euthymic (d = 0.14, p = 0.043), subclinical (d = 0.17, p = 0.001), manic (d = 0.25, p = 0.097), or depressed (d = -0.51, p < 0.001). However, moderator analyses should be treated with caution because of the observed confounding of moderators. Finally, none of the employed methods for publication-bias detection revealed any evidence for publication bias. We discuss further results, especially regarding the differences between subclinical and clinical samples.

Effects of the COVID-19 pandemic on reading performance of second grade children in Germany.

In education, among the most anticipated consequences of the COVID-19 pandemic are that student performance will stagnate or decline and that existing inequities will increase. Although some studies suggest a decline in student performance and widening learning gaps, the picture is less clear than expected. In this study, we add to the existing literature on the effects of the COVID-19 pandemic on student achievement. Specifically, we provide an analysis of the short- and mid-term effects of the pandemic on second grade reading performance in Germany using longitudinal assessments from over 19,500 students with eight measurement points in each school year. Interestingly, the effects of the pandemic established over time. Students in the first pandemic cohort even outperformed students from the pre-pandemic cohorts and showed a tendency towards decreased variances during the first lockdown. The second pandemic cohort showed no systematic mean differences, but generally had larger interindividual differences as compared to the pre-pandemic cohorts. While the gender achievement gap seemed unaffected by the pandemic, the gap between students with and without a migration background widened over time-though even before the pandemic. These results underline the importance of considering effects of the pandemic across cohorts, large samples, and fine-grained assessments. We discuss our findings considering the context-specific educational challenges and in terms of practical implications for teachers' professional development.

Quantity and Quality in Scientific Productivity: The Tilted Funnel Goes Bayesian.

The equal odds baseline model of creative scientific productivity proposes that the number of high-quality works depends linearly on the number of total works. In addition, the equal odds baseline implies that the percentage of high-quality works and total number of works are uncorrelated. The tilted funnel hypothesis proposes that the linear regression implied by the equal odds baseline is heteroscedastic with residual variance in the quality of work increasing as a function of quantity. The aim of the current research is to leverage Bayesian statistical modeling of the equal odds baseline. Previous work has examined the tilted funnel by means of frequentist quantile regression, but Bayesian quantile regression based on the asymmetric Laplace model allows for only one conditional quantile at a time. Hence, we propose additional Bayesian methods, including Poisson modeling to study conditional variance as a function of quantity. We use a classical small sample of eminent neurosurgeons, as well as the brms Bayesian R package, to accomplish this work. In addition, we provide open code and data to allow interested researchers to extend our work and utilize the proposed modeling alternatives.

Shaky Student Growth? A Comparison of Robust Bayesian Learning Progress Estimation Methods.

Monitoring the progress of student learning is an important part of teachers' data-based decision making. One such tool that can equip teachers with information about students' learning progress throughout the school year and thus facilitate monitoring and instructional decision making is learning progress assessments. In practical contexts and research, estimating learning progress has relied on approaches that seek to estimate progress either for each student separately or within overarching model frameworks, such as latent growth modeling. Two recently emerging lines of research for separately estimating student growth have examined robust estimation (to account for outliers) and Bayesian approaches (as opposed to commonly used frequentist methods). The aim of this work was to combine these approaches (i.e., robust Bayesian estimation) and extend these lines of research to the framework of linear latent growth models. In a sample of N = 4970 second-grade students who worked on the quop-L2 test battery (to assess reading comprehension) at eight measurement points, we compared three Bayesian linear latent growth models: (a) a Gaussian model, (b) a model based on Student's t-distribution (i.e., a robust model), and (c) an asymmetric Laplace model (i.e., Bayesian quantile regression and an alternative robust model). Based on leave-one-out cross-validation and posterior predictive model checking, we found that both robust models outperformed the Gaussian model, and both robust models performed comparably well. While the Student's t model performed statistically slightly better (yet not substantially so), the asymmetric Laplace model yielded somewhat more realistic posterior predictive samples and a higher degree of measurement precision (i.e., for those estimates that were either associated with the lowest or highest degree of measurement precision). The findings are discussed for the context of learning progress assessment.

Understanding ideational fluency as a survival process.

BACKGROUND: When students generate ideas, important inter-individual variance exists both in the quantity and the quality of ideas they are able to produce (e.g., perfectionists who have few highly creative ideas or mass producers who produce a lot of uncreative ideas). In educational psychology research on creativity, the relation between the quantity and quality of ideas has not been well understood, limiting progress in this area. AIMS: We conceptualized Ideational Fluency as a phenomenon that requires participants to 'survive' to produce more ideas, and where dropping out of the ideational process was analogous to 'dying'. Using this novel paradigm, we aimed to test the relations among Fluency (as a dependent variable); and creative Expertise, Originality and self-reported Personality attributes (as independent variables). SAMPLE AND METHOD: Participants were drawn from three groups: those with demonstrated expertise in stage or screen acting (n = 104); undergraduates being trained in the same domain (n = 100), and adults with no acting training or experience (n = 92). Participants responded to the Alternate Uses Task; Non-parametric and semi-parametric survival models were fit to their Ideational Fluency; and average and maximum Originality scores, as well as self-reported Personality attributes, were used as covariates. RESULTS: Across all participants, the Ideational Fluency survival function showed an S-shape, but the Expertise grouping interacted with that pattern. The survival rate of professional actors decreased more rapidly during the first few ideas, but after the 5th idea, professional actors displayed a clear advantage in survival rate. Participants who were less original on average but who showed a high maximum Originality, as well as those participants who reported more Assertiveness and less Industriousness, also survived further into the Ideational process. CONCLUSIONS: Contrary to our hypothesis, professional actors' advantage in Fluency did not manifest in the survival model until after the 5th idea generated. A quantity-quality trade-off was observed with average Originality being associated with shorter survival, but that trade-off was not observed with maximum Originality, which was associated with longer survival.

A three-wave longitudinal study on the relation between commuting strain and somatic symptoms in university students: exploring the role of learning-family conflicts.

BACKGROUND: We examine the role of learning-family conflicts for the relation between commuting strain and health in a sample of medical university students. The first goal of the study was to investigate the mediating role of learning-family conflicts. The second goal was to extend the temporal view on relations between study variables. Therefore, we differentiated long-term systematic change among variables over a period of two-years from a dynamic perspective with repeated commuting events on the individual level of analyses. METHODS: We applied a multilevel research design and collected survey data from 128 medical students on three points in time (N = 339 measurement points). Participants informed about commuting strain, learning-family conflicts, somatic symptoms, as well as commuting distance and time. RESULTS: Bayesian multilevel analyses showed that results differed with regard to level of analysis: while learning-family conflicts mediated the relation between commuting strain and somatic symptoms on a systematic aggregation-level perspective of analysis (indirect effect estimatebetween = 0.13, SE = .05, 95% CI [0.05; ∞), Evidence Ratio = 250.57), this was not the case on the dynamic event perspective (indirect effect estimatewithin = 0.00, SE = 0.00, 95% CI [- 0.01; ∞), Evidence Ratio = 0.84). CONCLUSIONS: We demonstrated that learning-family conflicts explain why commuting may have unfavorable effects on health for medical students. We also showed that it is the long-term commuting experience that is related to health complaints and not the single commuting event. This means that short-term deviations from general levels of commuting strain do not cause somatic symptoms, but general high levels of commuting strain do instead.

How Is Intelligence Test Performance Associated with Creative Achievement? A Meta-Analysis.

This paper presents a meta-analysis of the links between intelligence test scores and creative achievement. A three-level meta-analysis of 117 correlation coefficients from 30 studies found a correlation of r = .16 (95% CI: .12, .19), closely mirroring previous meta-analytic findings. The estimated effects were stronger for overall creative achievement and achievement in scientific domains than for correlations between intelligence scores and creative achievement in the arts and everyday creativity. No signs of publication bias were found. We discuss theoretical implications and provide recommendations for future studies.

Effects of Data-Based Decision-Making on Low-Performing Readers in General Education Classrooms: Cumulative Evidence From Six Intervention Studies.

In most general education classrooms in Germany, students with and without special educational needs are taught together. To support teachers in adapting instruction to these heterogeneous classrooms, we have developed learning progress assessment (LPA) and reading instructional materials, the Reading Sportsman (RS), in line with the theoretical framework of data-based decision-making, which has led to beneficial effects in several studies. However, data from these studies have not been analyzed to examine effects for low-performing readers. Data within and across six quasi-experimental studies conducted by our team in Grades 2 to 4 were reanalyzed to examine the effects of LPA on students whose reading fluency scores were at or below the 25th percentile (n = 1,346 students from 264 classes). In each study, students had been assigned to a control group (with business-as-usual instruction), an LPA group only, or an LPA-RS group (i.e., with teachers provided with LPA and the RS). Separate multilevel regression results were integrated by means of meta-analytical methods to investigate the consistency of results. Overall, findings from the single studies indicated no positive effects of LPA with or without the RS compared with the control group. The integrated analyses showed small positive effect trends on reading fluency and intrinsic reading motivation.

The Relationship between Intelligence and Divergent Thinking-A Meta-Analytic Update.

This paper provides a meta-analytic update on the relationship between intelligence and divergent thinking (DT), as research on this topic has increased, and methods have diversified since Kim's meta-analysis in 2005. A three-level meta-analysis was used to analyze 849 correlation coefficients from 112 studies with an overall N = 34,610. The overall effect showed a significant positive correlation of r = .25. This increase of the correlation as compared to Kim's prior meta-analytic findings could be attributed to the correction of attenuation because a difference between effect sizes prior-Kim vs. post-Kim was non-significant. Different moderators such as scoring methods, instructional settings, intelligence facets, and task modality were tested together with theoretically relevant interactions between some of these factors. These moderation analyses showed that the intelligence-DT relationship can be higher (up to r = .31-.37) when employing test-like assessments coupled with be-creative instructions, and considering DT originality scores. The facet of intelligence (g vs. gf vs. gc) did not affect the correlation between intelligence and DT. Furthermore, we found two significant sample characteristics: (a) average sample age was positively associated with the intelligence-DT correlation, and (b) the intelligence-DT correlation decreased for samples with increasing percentages of females in the samples. Finally, inter-moderator correlations were checked to take potential confounding into account, and also publication bias was assessed. This meta-analysis provides a comprehensive picture of current research and possible research gaps. Theoretical implications, as well as recommendations for future research, are discussed.

An Experimental Approach to Investigate the Involvement of Cognitive Load in Divergent Thinking.

Up to now, support for the idea that a controlled component exists in creative thought has mainly been supported by correlational studies; to further shed light on this issue, we employed an experimental approach. We used four alternate uses tasks that differed in instruction type ("be fluent" vs. "be creative") and concurrent secondary workload (load vs. no load). A total of 51 participants (39 female) went through all tasks and generated ideas for a total of 16 different objects; their responses were scored in terms of fluency (number of responses generated), creative quality, and flexibility. We did find, as expected, that the be-creative instruction resulted in fewer and more creative ideas, as well as more flexible idea sets, but neither of the expected interaction effects became significant. Specifically, fluency was not affected more strongly by secondary workload in the be-fluent instruction condition than in the be-creative instruction condition. Further, the performance drop evoked by the secondary workload was not stronger in the be-creative instruction condition compared to the be-fluent instruction condition when creative quality or flexibility were examined as dependent variable. Altogether, our results do not confirm that be-creative instructions involve more cognitive load than be-fluent instructions. Nevertheless, the analysis of the serial order effect and additional correlational examinations revealed some promising results. Methodological limitations which may have influenced the results are discussed in light of the inherent suspense between internal and external validity (i.e., most likely the applied self-paced dual-task approach increased external validity, but undermined internal validity) and potentially guide future research.

How Much g Is in the Distractor? Re-Thinking Item-Analysis of Multiple-Choice Items.

Distractors might display discriminatory power with respect to the construct of interest (e.g., intelligence), which was shown in recent applications of nested logit models to the short-form of Raven's progressive matrices and other reasoning tests. In this vein, a simulation study was carried out to examine two effect size measures (i.e., a variant of Cohen's ω and the canonical correlation RCC) for their potential to detect distractors with ability-related discriminatory power. The simulation design was adopted to item selection scenarios relying on rather small sample sizes (e.g., N = 100 or N = 200). Both suggested effect size measures (Cohen's ω only when based on two ability groups) yielded acceptable to conservative type-I-error rates, whereas, the canonical correlation outperformed Cohen's ω in terms of empirical power. The simulation results further suggest that an effect size threshold of 0.30 is more appropriate as compared to more lenient (0.10) or stricter thresholds (0.50). The suggested item-analysis procedure is illustrated with an analysis of twelve Raven's progressive matrices items in a sample of N = 499 participants. Finally, strategies for item selection for cognitive ability tests with the goal of scaling by means of nested logit models are discussed.

Construct validity of multiple mini interviews - Investigating the role of stations, skills, and raters using Bayesian G-theory.

Background: One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depend on specific social skills. Here, we provide a detailed investigation into the construct validity of MMIs, including the identification and quantification of performance facets (social skill-specific performance, station-specific performance, general performance) and their relations with other selection measures.Methods: We used data from three MMI samples (N = 376 applicants, 144 raters) that included six interview and role-play stations and multiple assessed social skills.Results: Bayesian generalizability analyses show that, the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. Furthermore, there were low or no correlations with other selection measures.Discussion: Our findings suggest that MMI ratings are less social skill-specific than originally conceptualized and are due more to general performance differences (across and within-stations). Future research should focus on the development of skill-specific MMI stations and on behavioral analyses on the extents to which performance differences are based on desirable skills versus undesired aspects.

Scrutinizing the basis of originality in divergent thinking tests: On the measurement precision of response propensity estimates.

BACKGROUND: The originality of divergent thinking (DT) production is one of the most critical indicators of creative potential. It is commonly scored using the statistical infrequency of responses relative to all responses provided in a given sample. AIMS: Response frequency estimates vary in terms of measurement precision. This issue has been widely overlooked and is addressed in the current study. SAMPLE AND METHOD: Secondary data analysis of 202 participants was performed. A total of 900 uniquely identified responses were generated on three DT tasks and subjected to a 1-parameter logistic model with a response as the unit of measurement which allowed for the calculation of response-level conditional reliability (and marginal reliability as an overall summary of measurement precision). RESULTS: Marginal reliability of response propensity estimates ranged from .62 to .67 across the DT tasks. Unique responses in the sample (the basis for the classic uniqueness scoring) displayed the lowest conditional reliability (across tasks: ≈ .50). Reliability increased nonlinearly as a function of both the frequency of occurrence predicted by the model (conditional reliability) and sample size (conditional and marginal reliability). CONCLUSIONS: This study indicates that the common practice of frequency-based originality scoring with typical sample sizes (e.g., N = 100 to N = 200) yields unacceptable levels of measurement precision (i.e., in particular for highly original responses). We further offer recommendations to mitigate the lack of measurement precision of frequency-based originality scores for DT research.

Revisiting dispersion in count data item response theory models: The Conway-Maxwell-Poisson counts model.

Count data naturally arise in several areas of cognitive ability testing, such as processing speed, memory, verbal fluency, and divergent thinking. Contemporary count data item response theory models, however, are not flexible enough, especially to account for over- and underdispersion at the same time. For example, the Rasch Poisson counts model (RPCM) assumes equidispersion (conditional mean and variance coincide) which is often violated in empirical data. This work introduces the Conway-Maxwell-Poisson counts model (CMPCM) that can handle underdispersion (variance lower than the mean), equidispersion, and overdispersion (variance larger than the mean) in general and specifically at the item level. A simulation study revealed satisfactory parameter recovery at moderate sample sizes and mostly unbiased standard errors for the proposed estimation approach. In addition, plausible empirical reliability estimates resulted, while those based on the RPCM were biased downwards (underdispersion) and biased upwards (overdispersion) when the simulation model deviated from equidispersion. Finally, verbal fluency data were analysed and the CMPCM with item-specific dispersion parameters fitted the data best. Dispersion parameter estimates indicated underdispersion for three out of four items. Overall, these findings indicate the feasibility and importance of the suggested flexible count data modelling approach.

A New Perspective on the Multidimensionality of Divergent Thinking Tasks.

In the presented work, a shift of perspective with respect to the dimensionality of divergent thinking (DT) tasks is introduced moving from the question of multidimensionality across DT scores (i.e., fluency, flexibility, or originality) to the question of multidimensionality within one holistic score of DT performance (i.e., snapshot ratings of creative quality). We apply IRTree models to test whether unidimensionality assumptions hold in different task instructions for snapshot scoring of DT tests across Likert-scale points and varying levels of fluency. It was found that evidence for unidimensionality across scale points was stronger with be-creative instructions as compared to be-fluent instructions which suggests better psychometric quality of ratings when be-creative instructions are used. In addition, creative quality latent variables pertaining to low-fluency and high-fluency ideational pools shared around 50% of variance which suggests both strong overlap, and evidence for differentiation. The presented approach allows to further examine the psychometric quality of subjective ratings and to examine new questions with respect to within-item multidimensionality in DT.

The Effect of Forced Language Switching during Divergent Thinking: A Study on Bilinguals' Originality of Ideas.

In the present study we experimentally manipulated language switching among bilinguals who indicated to be more or less habitual language switchers in daily life. Our aim was to investigate the impact of forced language switching on originality of produced ideas during divergent thinking, conditional on the level of habitual language switching. A sample of bilinguals was randomly assigned to perform alternate uses tasks (AUT's), which explicitly required them to either switch languages, or to use only one language while performing the tasks. We found that those who were instructed to switch languages during the AUT's were able to generate ideas that were on average more original, than those who were instructed to use only one language during the AUT's, but only at higher levels of habitual language switching. At low levels of habitual language switching, the effect reversed, and participants who were instructed to use only one language found ideas that were on average more original, than participants who were required to switch languages during the AUT's. Implications and limitations are discussed.